The current landscape and future of tablet-based cognitive assessments for children in low-resourced settings

Interest in measuring cognition in children in low-resourced settings has increased in recent years, but options for cognitive assessments are limited. Researchers are faced with challenges when using existing assessments in these settings, such as trained workforce shortages, less relevant testing stimuli, limitations of proprietary assessments, and inadequate parental knowledge of cognitive milestones. Tablet-based direct child assessments are emerging as a practical solution to these challenges, but evidence of their validity and utility in cross-cultural settings is limited. In this overview, we introduce key concepts of this field while exploring the current landscape of tablet-based assessments for low-resourced settings. We also make recommendations for future directions of this relatively novel field. We conclude that tablet-based assessments are an emerging and promising method of assessing cognition in young children. Further awareness and dissemination of validated tablet-based assessments may increase capacity for child development research and clinical practice in low-resourced settings.

a myriad of factors important for the application of computerized cognitive assessments across different cultural contexts. We also offer recommendations for the future output of tablet-based assessments, which may mediate the gap for use of cognitive testing within low-resourced settings. Overall, we determine that tablet-based assessments are a promising solution among these settings, and further insight and awareness of these tools may increase their utility.
(NEPSY) [18,19], the knock-tap tests [20], the Spin the Pots task [21], or parent report measures such as the Behavior Rating Inventory of Executive Function (BRIEF) [22]. These all vary in reliability and validity [13]. Most of these tools have been developed and validated within highresource settings, although some attempts have been made to develop versions for other settings. This can present further challenges, as most items in neuropsychological assessment tools are contextually related to settings where tools were developed and validated, therefore requiring lengthy and expensive adaptation process for use cross-culturally. This has been attempted in some settings, e.g., Kilifi, Kenya, where a battery was made for an African context; however, difficulties remain in capacity to scale-up training, application, and measurement [23]. Furthermore, tools created in high-resource contexts are often proprietary and the cost to administer in lowresourced settings can be prohibitive. There is, thus, a paucity of contextually relevant scalable cognitive assessment tools developed, validated, and used within low-resourced settings [24]. We define low-resourced settings as those with a population facing health inequities that negatively impact child development, typically in low-and middle-income countries.
Advances in technology, including the emergence of tablet computers, could be leveraged to aid in the scalable evaluation of cognition and particularly executive function, across a wide range of settings. Tablet computers are readily available, inexpensive, portable, and have functionality without internet. Paper-based childhood developmental assessment tools are increasingly being deployed on tablet computers using basic open data kits or similar programs. This includes the Malawi Developmental Assessment Tool (MDAT) [25] and the Global Scales of Early Development (GSED) [26]. While this standardizes administration and scoring by minimizing errors, these tools either rely on parent report, assuming parental knowledge of developmental milestones, or behavioral observations of children by non-specialists, which itself is resource-intensive and only partially addresses the workforce challenge.
In recent years, computerized tasks within tablet computers are increasingly being used to evaluate child performance directly. This has the potential to overcome many current challenges of measuring cognition in children globally. These tools permit the administration and scoring of a broad array of tasks that measure specific domains of cognitive functioning with minimal potential for error, while also having the potential to be gamified [27], increasing children's interest in these cognitive tasks. Furthermore, with cameras (to capture images and videos during task performance), accelerometers and gyroscopes (to estimate motion and force on the screen) [19,20], timers (to assess latency in responses) [28], and microphones (to capture audio), developers can tap into a wide range of child responses to assess their cognitive abilities [29]. Such nuanced variables are not feasibly evaluated with traditional pen-and-paper administration of cognitive assessment tools [30,31]. Because the cognitive tasks are administered with a tablet computer rather than skilled observation of child behavior, it requires very little workforce support for administration or scoring, enabling non-specialists with minimal training to administer tasks. While smartphone technology has similar strengths, the smaller screen size presents some limitations and, at this point, only a few cognitive tools are under development for smartphone use. In sum, tablet computers provide opportunities to perform high-quality cognitive assessments of children in a manner that is easily scalable and practical in low-resourced settings.
While this technology appears to be the future of cognitive testing, no known summary of these tools exists. This overview aims to summarize the current landscape of tablet-based cognitive assessment tools used in low-resourced settings that directly measure child performance. We pose considerations for use of these tools with the existing state of the evidence and make recommendations for ways in which this novel field can move forward. This overview is for those who may want to use tools for programmatic and research evaluation and therefore specifically targets information on the feasibility and validity of the tools in their current format, when used in low-resource settings.

Methodology for compiling tools
We identified tablet-based cognitive assessment tools by performing a general scoping literature search as well as through reaching out to international content experts working in the field. We aimed to identify peer-reviewed articles providing information on tablet-based cognitive assessment tools from electronic databases that included PubMed and Google Scholar identifying any publications from January 2000 to September 2021. The search terms used included "cognition", "executive function," "pediatrics," "tabled-based assessments," and "children." Inclusion criteria required that the assessment had to measure cognition was used in children under 18 years of age, was used in low-resourced settings, and was used on a tablet computer. Tools were excluded if they were only used in well-resourced settings or performed only on smartphones or laptop/desktop computers. Much of the information gathered required direct interviews with the developers and was not available through peer-reviewed articles. We therefore supplemented our literature search through general search engines, such as Google and Bing, and sought topic experts (i.e., those who lead and consult on projects measuring cognition in low-resourced settings, but do not themselves develop tools) to identify further tools. We emailed each application developer requesting an interview and further information about any other known tablet-based cognitive assessment tools used in lowresourced settings. Through our review of published papers, websites, and interviews with developers, we collected data on domains of cognition and executive function measured, time for training and administration, country of use and present adaptations, psychometric properties of each tool (as published to date), and feasibility of use.

Current landscape of tablet-based cognitive assessment tools for children
Numerous commercial and non-commercial tablet-based cognitive assessment tools have been developed in recent years. From our search, we identified 16 tools and described characteristics of each in Table 1 and Fig 1. Most tools were developed in North America, Europe, or Australia and initially used the English language, which was subsequently translated into other languages. Two tools had significant development work performed in India (DEEP and START), which included formative work in the community and iterative development of the user-interface through testing with non-specialist administrators and the target age range of children [32]. One tool has been further developed in Brazil (Educational Neuroscience App-Based Learning Environment (ENABLE). From interviews with the tool developers, the primary users of these tools, to date, are researchers, with administration being performed by research staff or community health workers. However, some tools, such as the Early Years Toolbox and NIH Toolbox, have also been used by educators and clinicians as well.
While nearly all commercial tools have age ranges that extend into adulthood, the noncommercial tools tend to have more narrow ranges, likely due to a target study population for whom the initial tool was developed. Most assessment tools contain batteries that consist of individual tests, and the length of the assessment is modifiable based on the number of tests selected. For example, Cantab and Cogstate each have over 16 tests that can be administered, depending on the needs of the user [33,34]. Most tools contain tests that each evaluate specific domains of cognition, often focusing on memory, attention, visual-spatial, and inhibition tasks ( Table 2). Babyscreen and the Minnesota Executive Function Scale (MEFS)/EFgo test the youngest age range of children (as young as 18 months of age), but only have 1 test administered measuring multiple sub-domains of cognition [35,36]. The MEFS/Efgo tool administers a test based similar to the Dimensional Change Card Sort task, aiming to measure inhibition,  cognitive flexibility, and memory [36]. This approach is linked to the challenges of tapping into specific, differentiated dimensions of cognitive tasks within these early ages [37].

Considerations for identifying the tool(s) of choice
While the concept and aims of these tablet-based cognitive assessment tools are often similar, their strengths and limitations are highlighted for potential users to consider while identifying an appropriate tool for their setting. Some limitations are inherent to the way tools are funded and designed. Non-commercial tools are often developed by academicians with grant funding. These are therefore typically dependent on this funding or other program fees to maintain and update applications along with the tablet operating system. While these tools are often open-source, and hence costs are generally low, researchers often must directly contact developers to arrange for an agreement for tool use. While this benefits smaller projects with budgetary constraints, the lack of available support can cause delays in their implementation or any adjustments required.
Specific tools also have their own strengths and limitations. The Early Years Toolbox is the only tool that is freely downloadable and has instructions on administration on its website, a great strength in its accessibility. However, due to the "open access" nature of the tool, this results in limited knowledge on the full extent on locations of use and the degree of success of cross-cultural administration. This can only be gleaned from published articles and reports of its use, which may be subject to publication bias. A summary of the strengths and limitations for each tool is included in Table 3. Associated psychometrics and additional information about these tools are detailed in the S1 File. We identified a number of factors we consider important when choosing a tool for use in a low-resourced setting for measuring cognition in early childhood. These include (a) ease of using technology to measure cognition in the early childhood; (b) validity in varying cultural contexts; (c) ease of adaptability for different settings; (d) overcoming workforce challenges; and (e) accessibility.

Use of technology to assess cognition in the early childhood
There is still debate as to whether young children can be assessed appropriately and reliably using technology in the early years. Within this review, most tablet-based tools start testing at 3 years and older. In high-resourced settings, evidence suggests that children as young as 2 years may appropriately engage with digital technologies [38,39]. However, at these early ages, the foundations of executive functioning and cognition are being laid, adding to the challenge of isolating those constructs for testing [40]. Measurement of cognition at under 3 years of age may therefore, still be best performed with in-person or parent-report assessment tools [41] or with neurophysiological tests or imaging that map neural correlates underlying cognitive functions [42]. Many tools, especially non-commercial tools, are gamified, featuring appealing and narrative graphics for children. While this aids in overcoming the challenge of engaging children in assessment tools, the bright and interesting images may inadvertently lead to an overestimation of the domain of attention and impulsivity [27].

Validity of tools for varying socioeconomic and cultural settings
An important first step in choosing a cognitive tool is to critically review its measurement properties (such as validity and reliability). The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines provide a framework to guide this [43] and supports the use of both qualitative (acceptability, face, and content validity) and quantitative aspects of validity (construct, cultural, and structural validity) alongside reliability (Fig 2). While nearly all the tablet-based cognitive tests included in this review (14/16) have some preliminary or published validity data (S1 File), the detail and rigor of this psychometric testing differs among tools. Some tools have attempted validation against a "gold standard." However, this is complex, particularly in international low-resourced settings since "gold standard assessment tools" are typically developed and validated in high-resourced western settings. Evaluating validity of tools can be particularly challenging within settings where children have limited daily exposure to technology or the items contained within the assessment tool. Tools are often developed for children with high exposure to technology, indicating that limited exposure may impact a child's ability to interact with the tool's content in the same meaningful way. However, evidence is emerging that suggests the lack of prior exposure to smart devices may not impact the accuracy with which tablet-based tools can measure cognition [28]. Indeed, a "rightsbased perspective" would argue that technology coupled with culturally neutral content can benefit children globally. However, our review has identified rigorous psychometric testing as a gap in this field, which warrants additional consideration as tablet-based tools are disseminated widely.
Notably, caution should be taken when interpreting the assessment scores for children who differ significantly, either by language, age, or culture, from the original normative population of the assessment tool. Certain aspects of cognition are more adaptive and highly prioritized in some settings compared to others, and thus, the normative range of scores is likely to differ across settings. Thus, issues will arise when one depends on the normative population data to determine cutoffs for deficits in cognition or to function as a comparison or control group.

Ease of adaptability of tools for different settings
The first generation of tablet-based assessment tools were programmed with a single language in mind, with no ability to adapt for different contexts or languages. More recent tools have built-in functionality to allow for easy adaptability between contexts and languages, as long as the appropriate adaptation methodologies are used. Some tools require very little language for their use. For example, Developmental Assessment on an E-Platform (DEEP) and Babyscreen have little to no language incorporated into the application, making linguistic adaptation an easy task [32,35]. Other tools contain more use of language, resulting in time-intensive adaptation processes involving forward and backward translations and cognitive interviewing to ensure face validity. This process takes multiple cycles and may require full re-programming of the test by assessment tool developers, the case for a recent cultural and linguistic adaptation of the NIH Toolbox for Kenya [44]. Even within a single country, certain images may need to be adjusted to accommodate the broadest range of cultures and contexts with the most familiar images used among cultures within the region.

Overcoming workforce challenges
A significant advantage of tablet-based cognitive assessment tools is their ability to simultaneously administer, score, and record, either within the tablet computer itself, or a cloud server. By removing the requirement for a psychologist or highly trained individual to administer the tool, these assessment tools can be administered by non-specialists on a scale far beyond traditional psychological tests. Most of the non-commercial (8/11) and commercial (4/5) tools require only a few hours of training for test administration, allowing for assessors from a broad array of educational and training backgrounds. Nearly all the tests do not require an assessor's observations as part of the scoring. One of the few exceptions is the NIH Toolbox's List Sorting task, which measures working memory and requires an assessor to input whether a verbally repeated series of words is correct or incorrect [45]. With minimal input from the assessors, these tablet-administered tools reduce the risk of bias and human error and make great strides in overcoming workforce challenges in low-resourced settings.

Accessibility
While few assessment tools are intended for commercial use, most have been created within academic settings and are freely available with open-source coding ( Table 1). Developers of some tools, e.g., the BENCI and Early Years Toolbox, stipulate at inception that their tools remain free of cost so that they could be easily utilized in low-resourced settings. The few academic, non-commercial tools that require some funding directly relate the costs to the time required to support the developer in adapting the code for a new setting and other factors, such as server maintenance.
For commercial tools, some have a published fee for their use. This can range from a few hundred dollars for a subscription, to many thousand dollars when paying for tests per participant. Costs may vary dependent on the number of tests and administrations included within the assessment battery. While costs associated with the use of commercial assessment tools may be a deterrent for many researchers in low-resourced settings, it ensures tools used are adequately maintained, with up-to-date information technology infrastructure and developers on staff to troubleshoot any data-related issues.

Future directions for tablet-based pediatric cognitive assessment tools
This review aimed to summarize the current landscape of tablet-based assessment tools that measure cognition in children, particularly those used in low-resourced settings, as a potential solution to poor healthcare infrastructure and workforce-related barriers. We identified 16 tools that ranged the full spectrum of possibilities: from open-sourced to proprietary, and from those in their early stages of piloting in one region to those with extensive validation in multiple countries. As this novel area of digital pediatric cognitive assessment tools emerge and are pushed up the global mHealth agenda [29,46], it is critical for users to consider a tool's psychometric properties, such as validity and reliability, before integrating it into clinical practice, research, or public health developmental surveillance systems. The COSMIN checklist provides useful guidance, not only to evaluate the validity and utility of a novel digital assessment tool, but also for developers to keep in mind while planning validation studies, or when describing the strengths and limitations of their tools to potential users [43].
In the absence of a true "gold-standard," tool developers should aim to generate local normative scores for these novel tools in the target populations, instead of benchmarking them to the available "gold-standards." The developers of the ENABLE and DEEP tool hope to develop a cloud database from their users, so that global sample "norms" can become available in open-source platforms (verbal communication from N. Pitchford and S. Bhavnani, respectively (February 2021)).
Additionally, while this review focuses on tablet-based assessment tools, we are keenly aware that smartphones represent the ultimate game changer in terms of achieving scale. The use of tablet-based assessment tools still requires a trained individual to go into the community or household for administration, whereas a smartphone-based assessment tool could potentially be downloaded onto a phone and then self-administered or administered by a parent within the home. Lead investigators of the NIH Toolbox are involved with the development of self-administered smartphone-based tests, Mobile Toolbox, which they hope will be available for public dissemination in 2023 [47]. Given the ubiquity of smartphones globally-over 5 billion subscribers, 70% of those residing in low-resourced settings-and cellular networks connecting 85% of the world's population [29], we believe the potential for these cognitive assessment tools to scale will vastly improve with smartphones administration. Tablet-based tools were the focus on this review as there are currently a number of well-studied and validated options available for use in low-resources settings. They also form a useful "bridge" between more expensive and potentially fragile laptop computers and a larger screen than mobile devices. It is clear, however, that smartphones are more accessible and cheaper. Presently, less information is available about applications on these devices for measuring cognition in children, smartphone-based cognitive assessments for other populations, such as adults with dementia, have recently been developed [48,49]. It is likely that this review will need updating considering the fast-moving field and will need to include a focus on smartphonebased tools once data on their validity are available. Over 80% of World Health Organization (WHO) member states use at least 1 mHealth initiative operationalized through smartphones [29], with features such as the use of videos and decision support systems, that have proven useful in improving maternal and child health in a variety of low-resourced settings [50]. Therefore, the integration of cognitive assessments in smartphones may be the next leap towards optimizing child development at scale across global settings.
The digital administration of these tools also makes it possible to sync data collected from different modalities, such as eye tracking and electroencephalography, to provide a deeper, more integrated level of evaluation. With advances in technology, further spurred by substantial increases in use of telemedicine during the Coronavirus Disease 2019 (COVID-19) pandemic, these assessment tools may facilitate the possibility of virtual, parent-led, in-home wellbaby checks in the future [51].
A forum to house the available digital tools, regularly updated to reflect the latest progress and associated data, would be valuable to stakeholders with interest in early childhood development. Such an effort has been initiated by the World Bank [52]. For this review, we aim to use an example of this and consolidate the information gleaned within the IMPACT Measures Tool database [53]. This online and open-source database is based on a research-driven scoring system that allows for the comparison of early childhood and parenting measures based on 4 categories (i.e., cost, usability, cultural relevance, and technical merit) [54]. Attending to the current landscape and new demands in low-resourced settings, we thought that the IMPACT database would benefit from the addition of digital measures, but other digital databases, as they emerge, should be considered.
This review has some limitations. Because we did not perform a formal systematic review, it is possible that we have missed some existing literature regarding tools currently being used in low-resourced settings outside of the network of investigators with whom we connected. Further, the dynamic and fast-paced nature of this emerging field would imply that new and updated tools are being added to the existing pool rapidly. Some have not yet been used in low-resourced settings and therefore will have not been included in this review. However, despite these limitations, we believe that this review of the existing tablet-based cognitive assessment tools adds great value, since the current landscape of available tools and possible future directions have not yet been summarized in this field.

Brief considerations for developers of digital cognitive evaluation tools
While this review is primarily targeted for users of tablet-based tools measuring cognition, we provide the following recommendations for tool developers. A primary consideration in the development of new tools should be their scalability. To benefit the large number of children who are faltering in cognition development within low-resourced settings, these tools must have the ability to adopt at scale within clinical practice and health systems. The mHealth Assessment and Planning for Scale (MAPS) toolkit published by the WHO provides a useful guide for iterative assessment of tool readiness for scaling up of mHealth tools, as well as providing strategies to address common barriers inherent in the pathway to scaling up [54]. Similarly, "Beyond Scale" launched by the Digital Impact Alliance is a free online course that highlights key challenges and solutions to scaling up mHealth solutions [55].
While understanding the importance of scalability, we also strongly recommend that tool developers closely collaborate with stakeholders from low-resourced settings-children and their families, researchers, health system staff, and policy makers-across all phases of development to ensure that the tool is designed optimally for its intended contexts, is affordable for scale-up, and able to be integrated within the health and educational sectors [56]. In partnership with key stakeholders, developers should also consider the local laws about data privacy and security. Most of the tools described in this review collect only de-identified study IDs, with password-protected storage on secure servers and varying levels of encryption and password keys. Ensuring security of data within the tablets and cloud storage is an essential feature for this technology.
And finally, cognitive tools only have the potential for meaningful impact when they are disseminated and being utilized. To ensure that information is readily available to others, we recognize that the addition of digital assessment tools in online repositories of child development measures is essential [53,57]. These efforts will help researchers and practitioners to understand and be informed as to what is best for them to use when selecting evaluation tools for their programs in this fast moving and dynamic field.

Conclusions
Tablet-based cognitive assessment tools may finally overcome the barriers of inadequate health systems that lead to poor measurement of child development outcomes in low-resourced settings. Data derived from these tools can then provide the foundation for drafting contextually relevant policies and practices-from the sub-local to the global-to optimize the developmental potential of all children globally.
Supporting information S1 File. This supplement contains 2 appendices. Appendix A is a table outlining the psychometric measurement properties of the tablet-based cognitive tools included in this review. Appendix B is another table that describes the additional neurodevelopmental domains evaluated by tablet-based assessments, with contact information of the developers. (DOCX)